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Preface 


This  report  is  one  of  a series  dealing  with  critical  performance 
requirements  for  different  occupational  groups. 

The  author  gratefully  acknowledges  the  subject  matter  advice  and 
technical  assistance  of  the  following  men  in  conducting  the  analyses: 

Chester  F.  Boratenski,  Consumer  & Marketing  Service 

Michiel  J.  Noordewier,  Forest  Service 

David  E.  Temple,  Consumer  & Marketing  Service 

Carl  Gorgovits,  Agricultural  Stabilization  6c 
Conservation  Service 

Wells  E.  Ludlow,  Rural  Electrification  Administration 
Paul  A.  Carroll,  Rural  Electrification  Administration 

The  technical  aspects  of  this  study  are  contained  in  technical 
appendices  which  are  available  from  the  research  files  upon  request. 
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SUMMARY  AND  HIGHLIGHTS 


This  report  describes  the  critical  incident  method  of  determining 
the  performance  requirements  for  accountants  and  budget  personnel  and 
for  developing  a personnel  performance  information  system  for  these 
occupations.  The  advantages  of  using  performance  records  as  source 
documents  for  many  kinds  of  personnel  actions  are  explained.  Various 
ways  in  which  management  can  use  the  information  contained  in  pools  of 
critical  incidents  to  aid  decision-making  in  such  areas  as  career 
programming,  spotting  training  needs,  evaluating  the  relevance  of 
training  courses,  and  appraising  the  effects  of  program  and  policy 
changes  are  also  illustrated. 
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To  answer  the  question,  "How  are  we  doing,"  management  depends  upon 
the  adequacy  of  its  performance  information  system.  In  any  occupational 
area,  if  we  have  inadequate  information  about  the  requirements  for  satis- 
factory performance  and  the  achievement  of  these  requirements,  we  cannot 
tell  how  well  we  are  doing. 

During  the  past  three  years  the  Personnel  Research  Staff  of  the 
U.  S.  Department  of  Agriculture  has  been  doing  research  aimed  at  improv- 
ing our  performance  information  systems.  The  following  is  a study  of 
critical  performance  requirements  for  the  1200-odd  Accounting  and  Budget 
personnel  of  the  Department, 

We  started  with  the  question:  "What  are  the  critical  requirements 
of  performance  in  this  occupation?"  We  got  the  facts  by  going  to  the 
people  directly  involved  in  doing  and  in  supervising  the  work.  We  asked 
each  of  them  to  describe  critical  incidents  that  they  had  observed. 

What  are  critical  incidents ? 

Critical  incidents  are  specific  actions  or  behaviors  that  have 
actually  been  observed.  They  do  not  involve  personality  labels  or  lists 
of  traits.  An  effective  critical  incident  leads  to  significantly  better 
than  average  accomplishment  of  a particular  aspect  of  a job,  assignment, 
or  responsibility.  An  ineffective  incident  leads  to  significant  delay, 
mistakes,  omissions,  or  lack  of  accomplishment.  One  might  say  that  an 
incident  is  called  critical  because  it  makes  a real  difference.  The 
judgments  are  based  upon  results--the  outcomes  of  peoples’  actions  and 
behaviors  and  the  products  of  their  work.  They  are  judgments  made  by 
people  at  various  levels,  but  always  by  the  people  in  the  best  position 
to  observe:  those  closest  to  the  operational  situation,  those  best 
situated  to  base  their  judgments  upon  the  objective  facts,  within  human 
limits  of  perceptual  fallibility. 

An  individual  critical  incident  is  not  an  evaluation  of  a person. 

It  is  an  observation  of  "what  happened,"  what  action  took  place,  and 
what  its  consequences  were. 

Each  person  was  asked  to  write  up  eight  incidents--two  effective 
and  two  ineffective  incidents  involving  "job  or  technical  competence" 
and  two  of  each  kind  involving  "working  with  people."  Altogether  we 
collected  about  5000  critical  incidents  from  Accounting  and  Budget 
personnel.  To  help  in  carrying  out  statistical  studies  of  group  data; 
we  also  asked  for  certain  additional  information,  but  complete  individual 
anonymity  was  assured. 
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Determining  critical  performance  requirements 

After  the  incidents  had  been  collected,  we  sorted  them  into  groups, 
according  to  the  different  critical  performance  requirements  that  they 
reflectedo  There  were  no  predetermined  categories.  Fifty-six  critical 
requirements  grew  out  of  the  content  of  the  incident  pool.  There  were 
slightly  more  of  these  requirements  for  the  ineffective  than  for  the 
effective  categories. 

The  related  critical  requirements  were  then  grouped  together  to 
identify  critical  performance  areas.  We  tested  our  classification  for 
objectivity  by  having  five  observers  independently  sort  a sample  of 
incidents.  The  average  agreement  among  observers  was  about  607o,  which 
was  deemed  sufficient  for  the  purposes  of  the  study.  This  procedure 
yielded  eight  performance  areas. 

The  performance  record 

A copy  of  the  form  developed  for  Accounting  and  Budget  personnel 
in  USDA  is  presented  in  Exhibit  1.  It  is  organized  by  performance  areas. 
Within  each  performance  area  the  related  critical  requirements  are 
listed-“ef f ective  behaviors  on  the  left  side,  ineffective  on  the  right. 
Spaces  are  provided  under  each  performance  area  for  the  supervisor  to 
make  notes  of  critical  incidents  at  the  time  they  occur.  He  will  have 
a Performance  Record  for  each  of  his  subordinates. 

Experience  in  industry  has  shown  that  on  the  average,  supervisors 
of  groups  of  up  to  thirty  employees  need  no  more  than  five  minutes  a day 
to  make  entries  for  the  whole  group.  This  is  so  for  two  reasons.  First, 
most  of  the  things  that  employees  do  fall  within  the  broad  range  of 
normal,  average,  typical,  satisfactory,  or  acceptable  performance. 
Critical  incidents  that  ’’make  a real  difference"  are  rarer.  Just  a few 
employees  are  likely  to  be  involved  on  any  given  day.  Some  days  there 
may  be  none.  Secondly,  most  of  the  basic  descriptive  material  is 
already  in  the  Performance  Record,  A short  note  in  the  appropriate 
performance  area  is  all  that  is  needed  to  insure  that  the  information 
will  not  be  forgotten  and  that  the  incident  can  be  recalled  later  on. 


Exhibit  1 
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I.  CARRYING  OUT  ASSIGNMENTS 
1.  PERFORMING  ASSIGNMENTS  ACCURATELY 


Ao  Detected  clerical  error. 

B.  Detected  error  in  procedure 
or  instructions.  C.  Completed 
task  with  minimal  information 
or  instructions. 


a.  Did  not  perform  specific  opera- 
tion or  task  correctly,  b.  Did  not 
follow  procedure  or  instructions, 

c.  Did  not  adequately  verify  and/or 
support  reports  or  statements, 

d.  Did  not  meet  deadline  or  report 
status  of  work,  e.  Did  not  follow 
through,  f.  Did  not  properly  safe- 
guard important  documents. 


DATE  ' 

ITEM; 

WHAT  HAPPENED 

DATE  ] 

ITEM 

WHAT  HAPPENED 

2/8 

c 

No  detail  on 

report  X,  redone 

2.  ASSUMING  RESPONSIBILITY  AND  RESPONDING  TO  NEED  FOR  EXTRA  EFFORT 


A,  Developed  new  procedure. 

B,  Demonstrated  alertness  and/or 
follow  through  in  unusual  cir- 
cumstances. C,  Carefully  planned 
work  assignment,  D,  Procured 
additional  information,  E,  Volun- 
tarily contributed  extra  effort 
to  complete  task. 


a.  Did  not  keep  informed  on  job 
activities  or  instructions,  b.  Did 
not  properly  utilize  time,  c.  Did 
not  meet  appointments,  d.  Did  not 
maintain  proper  personal  appear- 
ance, e.  Did  not  report  absence. 


DATE 

ITEM 

WHAT  HAPPENED 

DATE 

ITEM 

WHAT  HAPPENED 

6/5 

A 

Completed  report  '2.  in 

less  time  by  listing 

new  current  activities 

in  a separate  form. 

II,  WORKING  WITH  OTHERS 


3,  RELATIONSHIPS  WITH  OTHER  ORGANIZATIONS 


A,  Presented  information  tactfully. j a.  Did  not  use  tact  in  presenting 

B,  Gave  helpful  assistance  to  others.  information,  b.  Acted  inconsiderate 


toward  audite 

e’s  proeram. 

DATE 

ITEM 

WHAT  HAPPENED 

DATE  j ITEM 

WHAT  HAPPENED 

! 

i 

» 

1 

1 

j 
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4.  COOPERATING  AND  GETTING  ALONG  WITH  OTHERS 


a.  Refused  to  help  fellow  worker, 

b.  Acted  inconsiderately  toward 
others,  c.  Resisted  or  disobeyed 
supervisor,  d.  Did  not  coordinate 
with  other  group.  e.  Acted 
outside  scope  of  authority. 

f.  Used  offensive  language. 


DATE 

ITEM 

WHAT  HAPPENED 

DATE 

ITEM 

WHAT  HAPPENED 

' 

A.  Assisted  others  when  needed. 

B.  Stimulated  cooperation. 

C.  Made  skillful  group  presentation. 


III.  MANAGING  WORK  AND  SUPERVISING  OTHERS 


5.  PLANNING  AND  ORGANIZING  WORK 

A.  Provided  in  advance  for  contingencies]  a.  Set  up  inappropriate  assign- 
ments or  work  schedules, 
b.  Did  not  delegate  responsi- 
bilities appropriately. 


DATE 

ITEM 

WHAT  HAPPENED 

DATE 

ITEM 

WHAT  HAPPENED 

A;- 

6.  MOTIVATING  SUBORDINATES 


A.  Gave  praise  or  recognition  for 
superior  performance.  B.  Counseled 
privately  on  performance  when  needed, 
C.  Settled  difference  among 
employees.  D.  Considered  employee’s 
personal  problem.  E.  Developed 
teamwork  and  cooperation. 


a.  Criticized  in  presence  of 
others,  b.  Failed  to  discuss 
subordinate’s  performance, 

Co  Failed  to  find  cause  of  non- 
performance. do  Showed  favoritism 
to  subordinate,  e.  Failed  to 
maintain  cordial  relations  with 
subordinate.  f.  Acted  in 
arbitrary  or  dictatorial  manner. 


DATE 

ITEM  1 WHAT  HAPPENED 

DATE 

ITEM 

WHAT  HAPPENED 
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7.  TRAINING  AND  DEVELOPING  SUBORDINATES 


A.  Provided  clear  instructions 
and  training.  B.  Encouraged 
self-improvement . 

a.  Did  not  p 
adequate  ins 
to  try  new  a] 

rovide  training  or 
tructions.  b.  Refused 
pproaches . 

DATE 

ITEM 

WHAT  HAPPENED 

DATE 

ITEM 

WHAT  HAPPENED 

8.  MAINTAINING  COMMUNICATIONS 

A„  Notified  employee(s)  of  i a.  Did  not  notify  employeeCs)  of 

impending  change.  B.  Gave  impending  change,  b.  Did  not  clarify 

employee  opportunity  to  explain.  or  answer  questions,  c.  Did  not 

explain  reasons  for  decisions 
I affecting  employee,  d.  Did  not 

discuss  changes  in  employee’s  report. 


DATE 

ITEM 

WHAT  HAPPENED 

DATE 

ITEM  i WHAT  HAPPENED 

1 

i 

. . 

s 

I 

i 

1 

i 

IV.  MISCELLANEOUS 

(Identify  items  or  enter  additional  notes  or  incidents  that  have  not  been 
otherwise  classified). 


DATE 

ITEM 

WHAT  HAPPENED' 

DATE 

ITEM  ' WHAT  HAPPENED 

! 

. . 

- 

\ 

1 

(j 

' 

- - - _ _ _ _ i 

— 

. 
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Suppose,  for  exainple,  an  employee  fails  to  provide  supporting  detail 
for  a report  and  the  report  has  to  be  done  over.  Such  an  incident  falls 
in  performance  area  1_,  "Performing  Assignments  Accurately",  Within  that 
area  the  incident  fits  under  £,  on  the  right  hand  side  of  the  page.  As 
to  "What  Happened",  the  supervisor  might  merely  note  "No  detail  on  report 
X,  redone",  which  should  be  enough  to  recall  the  incident  to  his  mind  at 
some  later  time. 

Again,  suppose  that  the  employee's  performance  record  showed  the 
following  entry  under  "Completed  report  Z in  less  time  by  listing 

new  current  activities  in  a separate  form".  The  statement  of  critical 
•requirement  2A-wi 11  serve  to  remind  the  supervisor  that  this  incident 
involved  a new  twist  developed  by  this  employee.  In  both  examples  the 
relevant  information  is  neatly  summarized  in  a single  short  phrase. 


PUTTING  CRITICAL  INCIDENTS  TO  WORK 
Personnel  performance  information  for  deci s ion-making 

Up  to  this  point  we  have  concentrated  upon  providing  an  understand- 
ing of  what  critical  incidents  are,  and  how  they  are  used  to  define  the 
critical  performance  requirements  of  an  occupation  and  to  develop  a 
performance  record.  Hopefully,  experienced  managers  in  among  our  readers 
have  begun  to  perceive  the  applicability  and  utility  of  such  information 
at  many  points  across  a broad  range  of  management  activities. 

If  we  consistently  assemble  and  maintain  a store  of  information  on 
individuals*  performance  at  work,  we  will  have  available  the  essential 
facts  needed  to  arrive  at  any  of  the  administrative  personnel  decisions 
we  are  called  upon  to  make.  The  summation  of  the  pluses  and  minuses  of 
individual  performance  can  give  us  an  improved  comprehension  of  the 
requirements  that  most  influence  success  and  failure  in  achieving  our 
objectives.  We  will  reduce  the  burden  of  separately  maintaining  large 
overlapping  files  of  performance  information  for  each  administrative 
application  (e.g.  awards,  transfers,  promotions,  annual  ratings, 
training  needs). 

From  here  on  we  will  explore  in  greater  depth  some  specific  ways 
supervisors  and  managers  can  make  use  of  the  critical  incident  technique 
and  critical  incident  records. 

What  can  critical  incidents  tell  management? 

Our  answers  can  be  developed  in  two  areas*  (1)  individual  supervisor- 
subordinate  relations;  and  (2)  general  personnel  management  functions  and 
programs , 
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Individual  supervisor -subordinate  relations 

Accumulating  information  about  performance  in  the  critical  require- 
ments of  work  continually  rather  than  intermittently,  provides  a more 
objective  record  of  information.  The  pressures  to  *'mold’'  the  information 
are  less  than  when  it  must  be  pulled  from  your  head  only  when  a report 
on  an  individual  is  needed  for  some  specific  purpose.  This  is  so, 
because  the  ends  to  be  served  by  the  report  tend  to  determine  what 
information  will  be  retrieved  and  offered  on  that  occasion.  Contrary  to 
good  theory,  the  supervisor  in  practice  usually  makes  his  judgment  first, 
and  then  finds  "facts"  to  support  it,  instead  of  the  other  way  around. 

So,  if  all  performance  information  is  presented  as  part  of  administrative 
reports , there  remains  no  independent  source  of  data  collected  under 
"neutral"  conditions. 

A supervisor’s  note  in  a person’s  performance  record  is  neither  a 
punishment  nor  a reward.  The  performance  record  is  not  an  evaluation 
form,  nor  is  the  supervisor  attempting  to  evaluate  the  person  when  he 
makes  a note  for  the  record.  He  is  only  evaluating  the  quality  of  a 
specific  action  or  performance.  The  notes  are  merely  reminders  of  facts 
to  be  considered  in  talking  to  the  employee  about  his  work,  and  for 
reference  when  personnel  decisions  are  to  be  taken  later  on.  They  are 
as  much  the  property  of  the  employee  as  of  the  supervisor.  If  the 
employee  wants  to  use  these  notes  to  improve  himself,  they  are  available 
for  discussion  with  his  supervisor.  The  employee  makes  the  record.  The 
supervisor  only  keeps  it. 

Since  observations  are  noted  when  they  happen,  the  information  is 
less  subject  to  forgetting  and  errors  of  recall.  Since  the  critical 
incidents  are  identified  with  the  normal  work  situation  and  discussed 
when  they  happen,  both  parties  are  likely  to  develop  a better  balanced 
perspective. 

As  a consequence,  the  significance  of  individual  incidents  is  less 
likely  to  be  inflated  out  of  all  proportion  to  their  place  in  the  total 
picture  of  performance. 

Because  the  incidents  deal  with  observable  behavior  and  its 
results,  it  is  possible  to  discuss  performance,  and  even  to  evaluate 
it  on  a less  emotion-provoking  level  than  is  so  often  the  case  when 
inability  to  recall  specific  illustrative  incidents  lead  to  generalized 
and  questionable  personality  analysis. 

Furthermore,  the  summaries  of  an  individual’s  performance  history 
that  underlie  the  evaluations  and  decisions  needed  in-^perf ormance  review, 
career  planning,  promotion  rosters,  and  training  plans,  are  less  likely 


to  be  looked  upon  as  bureaucratic  procedures,  largely  divorced  from  the 
realities  of  day-to-day  experience  and  the  normal  relations  between  a 
man  and  his  boss.  This  can  mean  improvement  not  only  in  job  performance, 
attitudes,  and  common  understanding,  but  in  the  quality  of  supervision 
as  well. 

In  these  circumstances,  forms  and  procedures  are  developed  to 
facilitate  the  use  of  information,  but  the  forms  are  clearly  subordinated 
to  the  substance  of  the  information  system. 

As  a result,  the  occasions  when  formal  review  and  evaluation  take 
place  are  less  likely  to  be  regarded  as  uncomfortable  special  occasions 
during  which  both  parties  must  communicate  self-consciously,  with  their 
guards  up.  The  "summing  up"  is  more  likely  to  be  looked  upon  in  a 
healthy  fashion  as  the  natural  consequence  of  shared  experience,  and  as 
part  of  a jointly  held  aim  to  further  the  success  of  the  outfit  of  which 
both  are  a part. 

General  personnel  management  function  and  programs 

Apart  from  tho.se  situations  of  an  individual  nature,  the  pools  of 
critical  incidents  that  are  accumulated  offer  valuable  knowledge  to 
managers.  Given  supplementary  information  on  such  characteristics  of 
the  groups  involved  as  grade  level,  length  of  service,  age,  sex,  and 
occupational  specialty,  useful  comparisons  can  be  made.  Again  let  us 
look  at  some  examples. 

Taking  the  pulse  of  the  organ! zat ion 

What  are  the  areas  of  particular  strength  or  weakness  in  the 
functioning  of  the  organization  as  a whole?  Consider  Exhibit  2. 

Exhibit  2 is  based  upon  a sample  of  about  2100  incidents  drawn  from  the 
pool  of  5000.  We  see  that  relatively  high  concentrations  of  both 
effective  and  ineffective  incidents  occur  in  the  areas  of  "Performing 
Assignments  Accurately"  and  "Assuming  Responsibility  and  Responding  to 
Need  for  Extra  Effort."  However,  "Performing  Assignments  Accurately", 
shows  a higher  concentration  of  ineffective  than  effective  incidents, 

V7hi le  the  reverse  is  true  of  "Assuming  Responsibility  and  Responding 
to  Need  for  Extra  Effort".  The  area  of  "Motivating  Subordinates"  also 
has  a high  concentration  of  effective  and  ineffective  incidents.  On 
the  other  hand,  relatively  few  incidents  of  either  type  occur  in  areas 
of  "Planning  and  Organizing  Work"  and  "Maintaining  Communications". 

The  significance  for  management  of  a tabulation  like  this  is 
obvious.  It  pinpoints  the  areas  where  further  analysis  might  be 
profitable.  Take  just  one  example,  A comparison  of  the  distribution 
of  critical  incidents  with  present  statements  of  job  requirements  can 
show  whether  or  not  the  latter  need  to  be  revised. 


EXHIBIT  2 

PERCENT  OF  EFFECTIVE  AND  INEFFECTIVE  INCIDENTS  FOR 
COMBINED  SERIES  FOR  EACH  PERFORMANCE  AREA 
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Making  promotion  decisions 

How  do  the  critical  requirements  of  the  job  change  with  grade  level? 
Exhibit  3 shows  the  proportions  of  incidents  at  different  grade  levels  for 
each  performance  area.  At  the  lower  levels,  the  greatest  proportions  of 
incidents  involve  ’’Performing  Tasks  Accurately,”  and  ’’Assuming  Responsibil- 
ity and  Responding  to  Need  for  Extra  Effort”.  At  the  higher  levels, 
’’Motivating  Subordinates”,  ’’Training  and  Developing  Subordinates”,  and 
’’Maintaining  Communications”  account  for  the  greatest  proportion  of 
incidents.  No  surprise  here.  Nevertheless,  an  analysis  of  critical 
incidents  can  help  to  define  the  content  and  changes  in  work  that  occur 
at  these  different  levels,  thus  providing  management  with  more  specific 
information  than  might  otherwise  be  available. 

When  incidents  are  classified  and  tabulated  as  in  Exhibit  3,  it 
quickly  appears  that  the  supervisor  may  not  have  the  opportunity  to 
observe  behaviors  at  one  level  that  are  indicative  of  how  well  the  employee 
will  perform  at  a higher  level,  because  there  may  be  little  or  no 
opportunity  for  these  kinds  of  incidents  to  occur.  As  Exhibit  3 clearly 
shows,  ’’Planning  and  Supervising”  incidents,  which  preponderate  at  the 
higher  grade  levels  occur  very  infrequently  at  the  lower  levels.  Hence, 
in  making  a selection  for  a supervisory  job,  a promotion  board  might 
wish  to  supplement  performance  information  with  other  data  - for  example, 
tests  - to  get  a better  prediction  of  how  well  a person  might  be  expected 
to  function  as  a supervisor. 

Planning  job  rotation 

A job  rotation  plan  can  be  developed  by  classifying  the  incidents 
according  to  functional  areas.  This  would  enable  participants  in  the 
plan  to  gain  experience  in  certain  kinds  of  problem  situations  that  would 
enhance  their  performance  later  in  their  careers. 

Spotting  training  needs 

Just  as  an  individual  employee’s  performance  record  can  be  used  to 
diagnose  his  specific  training  needs,  so  too  can  a pool  of  critical 
incidents  collected  from  many  employees  point  up  training  needs  of  the 
group  as  a whole.  This  is  particularly  true  when  the  incidents  are 
classified  according  to  function  or  program  areas.  Inspection  of  the 
incidents  may  suggest  curriculum  content,  as  for  example,  certain  super- 
visory practices  that  need  to  be  emphasized  or  re-emphasized.  Indeed 
the  incidents  themselves  might  be  incorporated  as  study  material  in  a 
supervisory  training  program. 


PERCENT  OF  EFFECTIVE  AND  INEFFECTIVE  INCIDENTS  WITHIN  EACH 
GRADE  LEVEL  AND  SERIES  FOR  EACH  PERFORMANCE  AREA 
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Determining  relevance  of  training 

Critical  incidents  can  also  be  used  to  assess  and  validate  existing 
curriculum  content  by  checking  them  as  to  whether  the  relevant  skills  are 
"Taught"  or  "Not  Taught"  in  the  course. 

Evaluating  program  changes 

Still  another  potential  use  for  critical  incidents  is  the  evalua- 
tion of  program  and  policy  changes.  Suppose  that  a certain  organizational 
- or  procedural  change  is  introduced  in  order  to  alleviate  some  problem 
situation.  Incidents  collected  after  the  change  has  been  introduced  can 
be  compared  with  those  in  the  present  incident  pool,  and  a statement  as 
to  the  effects  of  the  change  then  can  be  made. 


CONCLUSION 

It  should  be  evident  by  now  that  critical  incidents  can  tell 
management  many  things  that  it  needs  to  know.  The  information  derivable 
from  an  analysis  of  critical  incidents  is  less  influenced  by  subjectivity 
and  bias,  because  it  is  based  upon  actually  observed  job  performance. 

At  the  same  time,  it  is  more  precise,  more  specific,  and  more  relevant 
to  the  requirements  of  the  job  than  information  obtained  by  more 
traditional  methods. 
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